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of 1988 (NELS) to examine student misbehavior in the context of individual, 
familial, and institutional influences. The NELS is the third longitudinal 
study of students and schools conducted by the National Center for Education 
Statistics, and the NELS surveys encompass not only students but also their 
parents, teachers, and school administrators. The purpose of the paper is to 
add to the experiential base of students using data collected by NELS. Latent 
factor structures of student misbehavior as well as variables that aid in the 
explanation of student misbehavior were derived through common factor 
analysis of selected NELS items. Factor structures were tested for internal 
consistency and generalizability by gender, race, and socioeconomic status. A 
series of exploratory univariate and multivariate logistic regression models 
were used to determine risk factors for and protective -factors against 
behavior problems. The NELS project provides a wealth of information with 
which to study student misbehavior, but it does pose certain limitations, 
including those of missing data, survey revisions, and survey breadth over 
depth. It was hypothesized that misbehavior would be distinct from substance 
abuse in high school, though strongly correlated, but scaling procedures 
yielded a misbehavior variable heavily defined by substance abuse. 

Misbehavior was inseparable from substance abuse, but substance abuse was not 
dependent on misbehavior for viability. One explanation for this finding may 
lie in the survey itself since misbehavior items were sparse and could have 
been explored more thoroughly. It was hypothesized that six variables would 
explain adolescent behavior, but only four reliable constructs were realized: 
student self-reported self-concept, peer influence, parent involvement, and 
administrator-reported school climate. It is hoped that this research can be 
used to design appropriate treatment and prevention programs for adolescents 
and to draw attention to the penetrating influence of a child’s friends on 
his or her behavior. It can also show the wealth of data collected and the 
usefulness of NELS data. (Contains 1 table and 10 references.) (SLD) 
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Abstract 

The National Education Longitudinal Study of 1988 (NELS) provides a wealth of 
information with which to study adolescent misbehavior, but it does pose certain challenges. 

Two problems arose due to the imposition of psychometric procedures on survey data in order to 
build reliable constructs. First, in large-scale surveys such as NELS, depth is often sacrificed for 
breadth in an effort to control costs while gathering information on the greatest number of topics. 
Creating psychometrically-sound measures was problematic in areas where survey items were 
sparse. Second, it is difficult to build sound constructs with missing data; with some NELS 
items, missing data were so prevalent that data imputation procedures would have created highly 
artificial structures. A third challenge encountered stems from revisions made across survey 
administrations. Even minor revisions in survey items present limitations when using NELS for 
longitudinal analyses. 

A challenge specific to researchers exploring adolescent behavior problems arose due to 
the composition of items addressing problem behaviors in school. Like many school surveys, the 
NELS project focuses primarily on externalizing behaviors and does not address some important 
internalizing behavior problems. Because underreactive behaviors are not measured by the NELS 
surveys, researchers using NELS data to study youth behavior problems should be cognizant of 
the limited scope of behaviors addressed. 

Even with these limitations, NELS does provide sufficient data to inform current thinking 
on many educational issues, including adolescent behavior problems. NELS also provides 
invaluable direction to educators on the relative importance of various influences of student 
misbehavior. 
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WHAT CAN NELS TELL US ABOUT ADOLESCENT BEHAVIOR PROBLEMS? 

(AND VICE VERSA) 

Introduction 

Regardless of who we are or where we live, we cannot escape the fact that many 
adolescents today are troubled. Recent violent incidents such as those in Colorado, Arkansas, and 
Kentucky have brought youth behavior to the forefront of the educational agenda. Though, 
student misbehavior, including both criminal and non-criminal activity, has long been a problem 
in our public schools. Clearly, schools operate in a larger environment of political and social 
forces that interact in a manner that can make the seemingly simple task of teaching and learning 
quite complex. Adding to this complexity, students bring to the classroom a set of experiences 
and values that are undeniably diverse. The resulting behavioral manifestations of these 
experiences and values are understandably quite diverse. With student indiscipline becoming an 
increasingly important problem in today’s schools, it is important that we understand, to the 
extent possible, the underlying influences of behavior. Our understanding of what shapes a 
student’s behavior is a first and necessary step towards developing programs that can positively 
affect indiscipline. 

This research uses the National Education Longitudinal Study of 1988 (NELS) to 
examine student misbehavior in the context of individual, familial, social, and institutional 
influences. NELS is the U.S. Department of Education’s National Center for Education Statistics 
third longitudinal study of students and schools (see NCES, 1996). The NELS project is notable 
not only because it follows a nationally representative sample of youth, but because it’s surveys 
encompass not only students, but also their parents, teachers, and school administrators. Several 
studies have used the NELS data to examine student behavior and related issues such as school 
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dropout, but none have truly exploited the longitudinal nature of NELS to study student 
misbehavior. While the primary objective of this research was to examine adolescent behavior, 
other important objectives were to investigate the usefulness of using NELS to look at behavior 
problems as well as to exploit the NELS survey data for use in psychometric research. The 
purpose of this paper is to add to the experiential base of studies using data collected by the 
National Education Longitudinal Study of 1988, by documenting issues that arose during sample 
construction and statistical analyses in hopes that future researchers may benefit from those 
experiences. 

Theoretical Framework 

Conceptual explanations for behavior problems range from genetic abnormalities 
inherent in the misbehaving individual to context-specific societal interpretations of behavior 
problems. While many argue that individuals should be treated for behavior problems - due to 
inadequate personality development, physical disorder, need deprivation, or inappropriate 
behavior reinforcements - some argue that discipline problems should be treated within the 
system and not within the individual. Ecological theory, in particular, hypothesizes that behaviors 
are a result of a complex interaction of many forces acting between an individual and his or her 
environment. This research uses the National Education Longitudinal Study of 1988 (NELS) to 
examine student misbehavior from an ecological perspective, that is, examining misbehavior 
from multiple perspectives including the individual, familial, social, and institutional influences. 

As stated in the preceding section, the purpose of this paper is to document the NELS- 
related learnings from doing this research, so that future NELS researchers may benefit from 
those experiences. Several studies have used NELS to explore student behavior. Weishew and 
Peng (1993) examined the base year NELS data to relate student misbehavior and a variety of 
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influences. Using the school as the unit of analysis and variables from all four NELS surveys 
(student, parent, teacher, and school administrator), the study found that schools with high 
misbehavior were more likely to be large, public, and urban; have substance abuse problems; 
have a high population of students from low socioeconomic status families; viewed as having a 
negative school climate and unfair discipline; and have teachers who had been teaching longer. 
The authors suspected that this last finding was related to teacher stress and burnout, though they 
were unable to test this hypothesis from the data. Finn (1993) used NELS to examine student at- 
riskness, including risk for academic failure and subsequent dropout. His study found that 
successful students are prepared for class and are not disruptive during class. 

Methods 

Latent factor structures of student misbehavior as well as variables that aid in the 
explanation of student misbehavior were derived through common factor analysis of selected 
NELS items. Factor structures were tested for internal consistency and generalizability by 
gender, race, and socioeconomic status. Internal consistency was measured using Cronbach’s 
alpha. Cross-year factor structures and subgroup factor structures were compared using Wrigley- 
Neuhaus coefficients of congruence (Guadagnoli & Velicer, 1991). The dependent variable and 
scaled explanatory variables were constructed through unit-weighting the factor solutions. Each 
variable was transformed into a T-score using area conversion. Modeling focused on the 15% of 
students on both extremes of the behavior T-score distribution and attempted to explain what 
factors characterized these students. 

A series of exploratory univariate and multivariate logistic regression models (see 
Hosmer and Lemeshow, 1989) were used to determine risk factors for and protective factors 
against behavior problems. The dependent variable, student misbehavior (derived through 
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common factor analysis as described above), was modeled for both the first and second NELS 
follow-ups, when most students were in 10 th and 12 th grade respectively. Models included 
individual (self-concept, attitudes towards school, reading achievement, mathematics 
achievement, race, sex, and age), familial (family structure, family socioeconomic status, and 
parental involvement), social (peer influence), and institutional variables (school type, school 
enrollment, community type, and school climate). From these exploratory models, three final 
models were constructed that parsimoniously explain 10 th and 12 th grade misbehavior. All 
analyses were performed at the individual student level. NELS weights were used in conjunction 
with the data to compensate for unequal probabilities of selection and to adjust for the effects of 
survey nonresponse. 

Because the NELS: 88 sample design was quite complex and included stratification, 
disproportionate sampling of certain strata, and clustered probability sampling, the statistical 
estimates calculated from this data are subject to sampling variability. To adjust for the design 
effect, WesVar Complex Samples Software was used to calculate standard errors and 
significance levels, while SAS was used to estimate regression coefficients and odds ratios. A 
Jacknife Repeated Replication (JRR) method was used to calculate appropriate standard errors 
based on the NELS complex sampling design. 

Results 

The National Education Longitudinal Study of 1988 provides a unique opportunity to 
comprehensively study educational issues, practices, and policies. As mentioned previously, 
NELS enables researchers to explore educational issues from an ecological perspective. This 
research examined the educational issue of student misbehavior by relating adolescent behavior 
problems to multiple influences. This report provides a documentation of the lessons learned 
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while using NELS. A separate manuscript documents the research findings relating individual, 
familial, social, and institutional influences to adolescent behavior. 

Following is a discussion of some of the lessons learned particular to researchers 
examining adolescent behavior problems. While the NELS project provides a wealth of 
information with which to study student misbehavior, it does pose certain limitations. This 
section will explore three such limitations, namely missing data, survey revisions, and survey 
breadth over depth. 

Lessons Learned. Perhaps one of the more interesting findings of this research was the 
composition of items signifying adolescent misbehavior. While it was hypothesized that 
misbehavior would be distinct from substance abuse in high school (though strongly correlated), 
scaling procedures yielded a misbehavior variable heavily defined by substance abuse. However, 
the relationship between misbehavior and substance abuse was not mutual in that misbehavior 
was inseparable from substance abuse, but substance abuse was not dependent upon misbehavior 
for viability. 

One explanation for this unequal relationship between misbehavior and substance abuse 
lies within the survey itself. While some of the more common general misbehavior actions and 
outcomes were represented in NELS (e.g., tardiness, truancy, and suspension), overall the 
misbehavior items were sparse and could have been explored more thoroughly with both 
students and teachers. For example, NELS did not address some noncriminal externalizing 
behaviors such as hyperactivity and it did not probe deeply into behaviors such as verbal 
aggression, vandalism, and weapons possession. Yet, the NELS surveys do adequately address 
adolescent substance abuse in the first and second follow-ups, with multiple items measuring 
alcohol, marijuana, and cocaine use. 




U 



8 



NELS and Adolescent Misbehavior 



8 



Further, with regard to adolescent behavior problems, NELS focuses primarily on 
externalizing behaviors and does not address some important internalizing behavior problems. 
Research in adolescent psychopathology has found two major dimensions of problem behavior in 
children and adolescents, underreactive behavior and overreactive behavior (McDermott, 1981, 
1982, 1986). Moreover, psychopathologists have identified many important manifestations of 
youth underreactive behavior problems, such as schizoid behavior, immaturity, obsessive 
behavior, depression, and withdrawal (Taublieb, 1997). Because these behaviors are not 
measured by the NELS surveys, researchers using NELS data to study youth behavior problems 
should be cognizant of the limited scope of behaviors addressed. 

While it was hypothesized that six variables would be formed that explained adolescent 
misbehavior, only four reliable constructs were realized: student reported self-concept, peer 
influence, and parent involvement, as well as administrator-reported school climate. Student 
reported self-concept was defined very broadly in NELS to include how students view 
themselves and their life, as the student survey only asked students to complete a general self- 
concept scale. The NELS student survey did not include items specifically related to academic 
self-concept, which is disappointing in that several researchers have found at least as strong an 
association between student misbehavior and academic self-concept as they have with general 
self-concept. 

On the other hand, the peer influence items included in NELS are primarily 
academically-based and inquire as to whether school-related matters are important or not to 
friends. During the factor analysis of explanatory items, the peer influence items began to 
separate into academic and social peer influence constructs. Unfortunately, there were too few 
social peer influence items asked of students to form a reliable construct, so the peer influence 
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construct is primarily academically-based peer influence. In future research, it might be 
interesting to examine if and how social and academic peer influences differentially relate to 
adolescent misbehavior. 

Like peer influence, parent involvement items were also academically oriented and 
included questions about how often the student and parent discuss educational issues and plans. 
Finally, the school climate construct is based primarily upon items dealing with the school 
administrator’s perception of how teachers interact with and view students. 

Missing Data. A total of 1,052 eighth-grade schools (about 3% of the nation’s 
approximately 38,000 eighth-grade schools) were chosen at random to participate in the 
NELS:88 program. From each school, approximately 26 students were randomly selected. Base 
year survey completion rates for NELS:88 yielded 1,035 schools and 24,599 students. The 
weighted aggregate of these 24,599 students is representative of the roster of approximately 
3,000,000 eighth grade students enrolled in public and private schools in 1988. 

During the first and second follow-ups of NELS:88, students were freshened to be 
representative of the students enrolled in the 10th grade in 1990 and the 12th grade in 1992, 
respectively. In each of the follow-ups, student, teacher, and school administrator surveys were 
administered. The parent survey was not repeated in 1990, but it did return in the 1992 follow- 
up. Additionally, a new component was added in both follow-ups to track school dropouts. 
Student transcripts also were collected in the second follow-up. 

For the purposes of this study, students who did not participate in all three school-year 
surveys - the 1988 base year, the 1990 first follow-up, and the 1992 second follow-up - were 
eliminated from the NELS:88 population. This sample of 16,489 students is referred to as the 88- 
92 panel. Further, in order to determine the effect of school size on misbehavior without 
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confounding it with possible effects resulting from school transfer, only panel students who were 
enrolled in the same high school during both the 1990 and 1992 measurement cycles were 
retained. Thus, high school dropouts and students who transferred schools between the 1990 and 
1992 surveys were not included in the analyses. Finally, students who did not have a complete 
set of associated teacher, parent, and school data at each relevant time point were eliminated. 
Students who met the above criteria were drawn from the NELS: 8 8 population; the resulting 
sample size was 3,919 students. NELS:88 weights were used in conjunction with the data to 
compensate for unequal probabilities of selection and to adjust for the effects of survey 
nonresponse. 

Several data imputation methods were investigated in an effort to salvage subjects; 
however, with some NELS items, missing data were so prevalent that data imputation procedures 
would have created highly artificial structures. The effects of missing data are apparent when 
parameters of demographic characteristics of the eighth grade population in 1988 are compared 
to the same parameters based on the weighted subsample. For instance, the sample results in a 
higher percentage of females than males. Further, whites are over-represented in the subsample 
while blacks and hispanics are underrepresented. Proportionally more students from the 
Northeast and Midwest and fewer students from the South and West were retained in the 
subsample than existed in the population. Students from high socioeconomic status families had 
less missing data than those from low socioeconomic status families. Table 1 provides a 
comparison of the relevant demographic characteristics of the eighth grade population in 1988 to 
those of the weighted subsample. 

Survey Revisions. Another limitation of using the NELS surveys stems from revisions 
made across survey administrations. Methodologically, NELS has not always been consistent 
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Table 1 



Demographic Characteristics of Eighth-Grade Population and Subsample 





Population / 


Subsample 


Gender 


Male 


50.2% 


44.0% 


Female 


49.8% 


56.0% 


Race 


Black 


13.2% 


6.1% 


Hispanic 


10.4% 


5.4% 


White 


71.7% 


84.3% 


Region 


Northeast 


19.2% 


24.7% 


Midwest 


25.7% 


33.6% 


South 


35.7% 


29.5% 


West 


19.4% 


12.2% 


SES 


Low 


23.6% 


15.0% 


Medium Low 


24.9% 


23.1% 


Medium High 


25.5% 


27.4% 


High 


26.0% 


34.5% 



Note. Population parameters are based on a nationally representative weighted sample of 
16,489 eighth grade students in 1988. Subsample parameters are based on a weighted subsample 
of 3,919 students extracted from the full sample. 
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across measurement points. Even minor revisions in survey items present limitations when using 
NELS for longitudinal analyses. Specifically, several items on the base year survey were altered 
in the 1990 and 1992 administrations and many additional items were added at subsequent 
measurement points. For example, neither peer influence items nor questions pertaining to 
substance abuse were included on the base year survey. Such changes make it difficult to include 
the 1988 survey data in longitudinal analyses of student misbehavior. 

Further, many of the items included in the 1988 student survey are different from those 
included in the 1990 and 1992 surveys, making longitudinal analyses with these items difficult if 
not impossible. For example, in several circumstances the response point-scale of an item was 
changed between follow-ups and in some instances the stem of the item was altered. Thus, 
approximately 75 items that are common across the 1990 and 1992 student questionnaires were 
used in the analyses for this research. Twenty of these items pertain to student misbehavior. 
About 55 items address issues such as student participation in school activities, self-concept, 
peer influence, attitudes towards school, and parental involvement. About half of these items are 
identical to base year items and were used to relate base year measures to high school 
misbehavior. Additionally, standardized achievement test scores in reading and mathematics 
based on the cognitive test battery administered during the base year and at the first and second 
follow-ups were used in the analyses. 

Survey Breadth Over Depth. This research attempted to bridge the fields of survey 
research and psychometrics by deriving psychometrically viable constructs from survey data. It 
is recognized that surveys are often not designed for the measurement of psychometrically-sound 
constructs, but rather they are more likely employed as an applied tool for gathering information 
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about society and social programs. Regardless, the unreliability in single-item measures often 
produces results that are uninterpretable or misleading (Rossi, Wright, & Anderson, 1983). 

Even though developers of large-scale surveys such as NELS are experts in the field of 
survey design and have experience in phrasing survey items so as to minimize unreliability, the 
reliability of individual item responses is still unacceptable. It is possible to further improve 
measurement by using procedures such as item factoring to derive psychometrically sound 
variables, however this process becomes problematic when there are too few survey items 
surrounding a topic to create a viable construct. Evidence that the NELS developers recognized 
this limitation of surveys is apparent in several areas, such as self-concept and cognitive testing. 
Though, as is often times done in large surveys with broad-scale administration, depth is 
sacrificed for breadth in an effort to control costs while gathering the information on the greatest 
number of topics. Unfortunately the measurement of adolescent externalizing behaviors was one 
topic where depth was sacrificed for survey breadth while the measurement of internalizing 
adolescent behaviors was an area that was sacrificed almost in entirety. Again, this is not to 
necessarily criticize survey developers, it is simply to heighten awareness of, in general, the 
limitations involved in trying to psychometrically utilize survey instrumentation and more 
specifically, the limitations of using NELS to study student behavior problems. 

Discussion 

The composition of adolescent misbehavior as found in this research is compelling in that 
it is representative of a common view of problem behaviors in school. In measuring youth 
behavior problems, survey developers are not always cognizant of the research base on child and 
adolescent psychopathology, but rather refer to the common view of behavior problems. That is, 
while externalizing behavior problems may be the ones that schools address more frequently and 
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that are publicized in the media, internalizing behavior problems are equally as important and 
perhaps more important for future well-being when not identified and addressed in adolescence. 
Moreover, externalizing behavior problems are more prevalent in adolescent boys (McDermott 
& Weiss, 1995), as confirmed by this study. Yet, by focusing only on the typical problems that 
boys face during adolescence, instruments used to identify behavior problems are doing a 
disservice to many adolescents, especially adolescent girls, who more often manifest disturbance 
through withdrawal, depression, and indifference. It is plausible that adolescent substance abuse 
is comorbid with some internalizing behaviors (similar to the relationship between substance 
abuse and externalizing behaviors), though unfortunately this association could not be tested 
using NELS data. In future survey work, it is important that practitioners and researchers alike 
exploit the knowledge base in psychopathology in order to address behaviors in both sufficient 
depth and sufficient breadth. 

As NCES embarks on the design and implementation of future longitudinal studies, it is 
recommended that survey developers pay special attention to item wording, item longevity, and 
response options within and across survey administrations. While NELS items pertaining to self- 
concept were stable from the base year through the second follow-up, inconsistencies were found 
for items relating to many other constructs. For example, items addressing student-reported 
attitudes towards school and peer influence measured these variables much more thoroughly in 
the second follow-up survey than in previous survey administrations. In addition, administrator- 
reported school climate items underwent a point-scale change between the first and second 
follow-ups. Finally, questions addressing student participation in school activities were not 
consistently worded across survey waves. Survey changes such as these pose difficulties to 
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researchers as they attempt to explore educational issues across time points and as they prepare 
data for analyses. 

Documents compiled by NCES such as the instrumentation manuals are extremely 
beneficial to researchers. Specifically, the intracohort analyses provided in the NCES 
documentation are invaluable to researchers who want address research questions longitudinally. 
However, such documents can also be misleading if they do not reveal critical intricacies such as 
minor stem or anchor alterations. In addition to paying special attention to items across surveys 
in an effort to minimize inconsistencies, it is also recommended that NCES develop item maps 
(see Boruch & Terhanian, 1998) that can aid researchers in identifying item consistencies as well 
as irregularities across survey administrations. 

Educational Importance of the Study 

It is hoped that the findings from this research can be used to design appropriate 
treatment and prevention programs for adolescents, as well as draw attention to the penetrating 
influence of a child’s friends on his or her behavior. It is also hoped that this research can aid 
future researchers in understanding the advantages and limitations of using NELS to study 
student behavior problems and the ramifications of imposing psychometric methodologies on 
survey research. Finally, it is hoped that this research can raise awareness of the wealth of 
already collected, easily accessible data available to all researchers as well as the endless array of 
research questions that might be explored through these data. 
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